Increasing the Robustness of Biometric Templates for Dynamic Signature Biometric Systems

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. R. Tolosana, R. Vera-Rodriguez, J. Ortega-Garcia and J. Fierrez, "Increasing the robustness of biometric templates for dynamic signature biometric systems," Security Technology (ICCST), 2015 International Carnahan Conference on, Taipei, 2015, pp. 229-234. doi: 10.1109/CCST.2015.7389687Due to the high deployment of devices such as smartphones and tablets and their increasing popularity in our society, the use of biometric traits in commercial and banking applications through these novel devices as an easy, quick and reliable way to perform payments is rapidly increasing. The handwritten signature is one of the most socially accepted biometric traits in these sectors due to the fact that it has been used in financial and legal transitions for centuries. In this paper we focus on dynamic signature verification systems. Nowadays, most of the state-of-the-art systems are based on extracting information contained in the X and Y spatial position coordinates of the signing process, which is stored in the biometric templates. However, it is critical to protect this sensible information of the users signatures against possible external attacks that would allow criminals to perform direct attacks to a biometric system or carry out high quality forgeries of the users signatures. Following this problem, the goal of this work is to study the performance of the system in two cases: first, an optimal time functions-based system taking into account the information related to X and Y coordinates and pressure, which is the common practice (i.e. Standard System). Second, we study an extreme case not considering information related to X, Y coordinates and their derivatives on the biometric system (i.e. Secure System), which would be a much more robust system against attacks, as this critical information would not be stored anywhere. The experimental work is carried out using e-BioSign database which makes use of 5 devices in total. The systems considered in this work are based on Dynamic Time Warping (DTW), an elastic measure over the selected time functions. Sequential Forward Features Selection (SFFS) is applied as a reliable way to obtain an optimal time functions vector over a development subset of users of the database. The results obtained over the evaluation subset of users of the database show a similar performance for both Standard and Secure Systems. Therefore, the use of a Secure System can be useful in some applications such as banking in order to avoid the lost of important user information against possible external attacks.This work was supported in part by the Project Bio-Shield (TEC2012-34881), in part by Cecabank e-BioFirma Contract, in part by the BEAT Project (FP7-SEC-284989) and in part by Catedra UAM-Telefonica

Update Strategies for HMM-Based Dynamic Signature Biometric Systems

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. [R. Tolosana, R. Vera-Rodriguez, J. Ortega-Garcia and J. Fierrez, "Update strategies for HMM-based dynamic signature biometric systems," Information Forensics and Security (WIFS), 2015 IEEE International Workshop on, Rome, 2015, pp. 1-6. doi: 10.1109/WIFS.2015.7368583Biometric authentication on devices such as smart- phones and tablets has increased significantly in the last years. One of the most acceptable and increasing traits is the handwrit- ing signature as it has been used in financial and legal agreements scenarios for over a century. Nowadays, it is frequent to sign in banking and commercial areas on digitizing tablets. For these reasons, it is necessary to consider a new scenario where the number of training signatures available to generate the user template is variable and besides it has to be taken into account the lap of time between them (inter-session variability). In this work we focus on dynamic signature verification. The main goal of this work is to study system configuration update strategies of time functions-based systems such as Hidden Markov Model (HMM) and Gaussian Mixture Models (GMM). Therefore, two different cases have been considered. First, the usual case of having an HMM-based system with a fixed configuration (i.e. Baseline System). Second, an HMM-based and GMM-based sys- tems whose configurations are optimized regarding the number of training signatures available to generate the user template. The experimental work has been carried out using an extended version of the Signature Long-Term database taking into account skilled and random or zero-effort forgeries. This database is comprised of a total of 6 different sessions distributed in a 15-month time span. Analyzing the results, the Proposed Systems achieve an average absolute improvement of 4.6% in terms of EER(%) for skilled forgeries cases compared to the Baseline System whereas the average absolute improvement for the random forgeries cases is of 2.7% EER. These results show the importance of optimizing the configuration of the systems compared to a fixed configuration system when the number of training signatures available to generate the user template increases.This work was supported in part by the Project Bio-Shield (TEC2012-34881), in part by Cecabank e-BioFirma Contract, in part by the BEAT Project (FP7-SEC-284989) and in part by Catedra UAM-Telefonica

Preprocessing and feature selection for improved sensor interoperability in online biometric signature verification

Under a IEEE Open Access Publishing Agreement.Due to the technological evolution and the increasing popularity of smartphones, people can access an application using authentication based on biometric approaches from many different devices. Device interoperability is a very challenging problem for biometrics, which needs to be further studied. In this paper, we focus on interoperability device compensation for online signature verification since this biometric trait is gaining a significant interest in banking and commercial sector in the last years. The proposed approach is based on two main stages. The first one is a preprocessing stage where data acquired from different devices are processed in order to normalize the signals in similar ranges. The second one is based on feature selection taking into account the device interoperability case, in order to select to select features which are robust in these conditions. This proposed approach has been successfully applied in a similar way to two common system approaches in online signature verification, i.e., a global features-based system and a time functions-based system. Experiments are carried out using Biosecure DS2 (Wacom device) and DS3 (Personal Digital Assistant mobile device) dynamic signature data sets which take into account multisession and two different scenarios emulating real operation conditions. The performance of the proposed global features-based and time functions-based systems applying the two main stages considered in this paper have provided an average relative improvement of performance of 60.3% and 26.5% Equal Error Rate (EER), respectively, for random forgeries cases, compared with baseline systems. Finally, a fusion of the proposed systems has achieved a further significant improvement for the device interoperability problem, especially for skilled forgeries. In this case, the proposed fusion system has achieved an average relative improvement of 27.7% EER compared with the best performance of time functions-based system. These results prove the robustness of the proposed approach and open the door for future works using devices as smartphones or tablets, commonly used nowadays.This work was supported in part by the Project Bio-Shield under Grant TEC2012-34881, in part by Cecabank e-BioFirma Contract, and in part by Catedra UAM-Telefonic

BehavePassDB: Public Database for Mobile Behavioral Biometrics and Benchmark Evaluation

Mobile behavioral biometrics have become a popular topic of research, reaching promising results in terms of authentication, exploiting a multimodal combination of touchscreen and background sensor data. However, there is no way of knowing whether state-of-the-art classifiers in the literature can distinguish between the notion of user and device. In this article, we present a new database, BehavePassDB, structured into separate acquisition sessions and tasks to mimic the most common aspects of mobile Human-Computer Interaction (HCI). BehavePassDB is acquired through a dedicated mobile app installed on the subjects devices, also including the case of different users on the same device for evaluation. We propose a standard experimental protocol and benchmark for the research community to perform a fair comparison of novel approaches with the state of the art1. We propose and evaluate a system based on Long-Short Term Memory (LSTM) architecture with triplet loss and modality fusion at score levelThis project has received funding from the European Unions Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement no. 860315, and from Orange Labs. R. Tolosana and R. Vera-Rodriguez are also supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER

e-BioSign Tool: Towards Scientific Assessment of Dynamic Signatures under Forensic Conditions

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. R. Vera-Rodriguez, J. Fierrez, J. Ortega-Garcia, A. Acien and R. Tolosana, "e-BioSign tool: Towards scientific assessment of dynamic signatures under forensic conditions," 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS), Arlington, VA, 2015, pp. 1-6. doi: 10.1109/BTAS.2015.7358756This paper presents a new tool specifically designed to carry out dynamic signature forensic analysis and give sci- entific support to forensic handwriting examiners (FHEs). Traditionally FHEs have performed forensic analysis of paper-based signatures for court cases, but with the rapid evolution of the technology, nowadays they are being asked to carry out analysis based on signatures acquired by digi- tizing tablets more and more often. In some cases, an option followed has been to obtain a paper impression of these sig- natures and carry out a traditional analysis, but there are many deficiencies in this approach regarding the low spa- tial resolution of some devices compared to original off-line signatures and also the fact that the dynamic information, which has been proved to be very discriminative by the bio- metric community, is lost and not taken into account at all. The tool we present in this paper allows the FHEs to carry out a forensic analysis taking into account both the tra- ditional off-line information normally used in paper-based signature analysis, and also the dynamic information of the signatures. Additionally, the tool incorporates two impor- tant functionalities, the first is the provision of statistical support to the analysis by including population statistics for genuine and forged signatures for some selected features, and the second is the incorporation of an automatic dy- namic signature matcher, from which a likelihood ratio (LR) can be obtained from the matching comparison between the known and questioned signatures under analysis.This work was supported in part by the Project Bio-Shield (TEC2012-34881), in part by Cecabank e-BioFirma Contract, in part by the BEAT Project (FP7-SEC-284989) and in part by Catedra UAM-Telefonica

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

This chapter describes a DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. This chapter explores to what extent rPPG is useful for the detection of DeepFake videos. We analyze the recent fake detector named DeepFakesON-Phys that is based on a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames, analyzing and combining both sources to better detect fake videos. DeepFakesON-Phys has been experimentally evaluated using the latest public databases in the field: Celeb-DF v2 and DFDC. The results achieved for DeepFake detection based on a single frame are over 98% AUC (Area Under the Curve) on both databases, proving the success of fake detectors based on physiological measurement to detect the latest DeepFake videos. In this chapter, we also propose and study heuristical and statistical approaches for performing continuous DeepFake detection by combining scores from consecutive frames with low latency and high accuracy (100% on the Celeb-DF v2 evaluation dataset). We show that combining scores extracted from short-time video sequences can improve the discrimination power of DeepFakesON-PhysThis work has been supported by projects: PRIMA (H2020-MSCA-ITN2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (MINECO/FEDER RTI2018-101248-B-I00), and COST CA16101 (MULTI-FORESEE). J. H.-O. is supported by a PhD fellowship from UA

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection

© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksThe availability of large-scale facial databases, together with the remarkable progresses of deep learning technologies, in particular Generative Adversarial Networks (GANs), have led to the generation of extremely realistic fake facial content, raising obvious concerns about the potential for misuse. Such concerns have fostered the research on manipulation detection methods that, contrary to humans, have already achieved astonishing results in various scenarios. In this study, we focus on the synthesis of entire facial images, which is a specific type of facial manipulation. The main contributions of this study are four-fold: i) a novel strategy to remove GAN 'fingerprints' from synthetic fake images based on autoencoders is described, in order to spoof facial manipulation detection systems while keeping the visual quality of the resulting images; ii) an in-depth analysis of the recent literature in facial manipulation detection; iii) a complete experimental assessment of this type of facial manipulation, considering the state-of-the-art fake detection systems (based on holistic deep networks, steganalysis, and local artifacts), remarking how challenging is this task in unconstrained scenarios; and finally iv) we announce a novel public database, named iFakeFaceDB, yielding from the application of our proposed GAN-fingerprint Removal approach (GANprintR) to already very realistic synthetic fake images. The results obtained in our empirical evaluation show that additional efforts are required to develop robust facial manipulation detection systems against unseen conditions and spoof techniques, such as the one proposed in this studyThis work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), BioGuard (Ayudas Fundación BBVA a Equipos de Investigación Cientíifica 2017), Accenture, by NOVA LINCS (UIDB/04516/2020) with the financial support of FCT - Fundação para a Ciência e a Tecnologia, through national funds, and by FCT/MCTES through national funds and co-funded by EU under the project UIDB/EEA/50008/202

Estudio de interoperabilidad en sistemas biométricos de firma manuscrita dinámica

En este proyecto se estudian, implementan y evalúan sistemas de reconocimiento biométrico de firma dinámica en presencia de firmas procedentes de distintos dispositivos de captura. Para llevarlo a cabo se han utilizado y comparado diversas técnicas del estado del arte en reconocimiento de firma. A su vez se ha realizado un estudio de las diversas técnicas de normalización de datos usadas en el ámbito de reconocimiento biométrico para conseguir un sistema robusto independientemente del dispositivo de captura utilizado para entrenar o testear el sistema. Como punto de partida del proyecto se ha realizado un estudio de las diferentes técnicas que han ido marcando el estado del arte, haciendo especial hincapié en los sistemas basados en características globales y en los sistemas basados en características locales o funciones temporales. Una vez entendido el estado del arte desde el punto de vista teórico, el siguiente paso ha sido definir la tarea sobre la que se han evaluado las diferentes técnicas. Históricamente, la tarea principal en evaluaciones de firma dinámica ha consistido en entrenar y testear el sistema con firmas obtenidas de un mismo dispositivo de captura (sin interoperabilidad). En la tarea que hemos llevado a cabo para la realización de este proyecto disponemos de firmas de un mismo usuario obtenidas con distintos dispositivos de captura. Para la parte experimental se han llevado a cabo tres etapas. Durante la primera etapa el objetivo fue evaluar el rendimiento del sistema de verificación de firma dinámica con y sin interoperabilidad siguiendo el protocolo de las evaluaciones Biosecure Multimodal Evaluation Campaign (BMEC). En la segunda etapa se estudió y se aplicó al sistema con interoperabilidad técnicas de normalización presentes en el ámbito de reconocimiento biométrico con el objetivo de conseguir un rendimiento lo más parecido posible al sistema sin interoperabilidad. En la última etapa se ha aplicado técnicas de selección y fusión de características para obtener un sistema global robusto ante firmas de test provenientes de distintos dispositivos de captura. Finalmente, se presentan las conclusiones extraídas a lo largo de este trabajo, así como las posibles líneas de trabajo futuro.This thesis is focused on the development of a robust dynamic signature veri cation system dealing with interoperability (e.g. PDA, pen tablet). The goal of this project is to obtain an acceptable recognition performance when it is trained and tested with signatures coming from di erent capturing devices. As an starting point, an exhaustive study of the state-of-the-art on dynamic signature veri - cation techniques has been conducted. In this sense, particular attention was paid on recognition systems but also on feature selection algorithms (SFFS), data normalization, score normalization techniques and fusion of global and local systems. It is worth noting that little attention has been paid in the literature to the problem of interoperability in the eld of dynamic signature recognition. In order to carry out an evaluation of the recognition systems, we use the data and experimental protocols de ned in Biosecure Multimodal Evaluation Campaign (BMEC). The experiments conducted were divided in three phases. First we evaluated the recognition of the systems with and without interoperability. Secondly we studied data normalization methods in order to improve the similarity among signatures from di erent devices. Finally the goal was to obtain a global robust system with acceptable performance in scenario with signatures coming from di erent devices. To achieve this, feature selection algorithms and fusion methods were used

Aproximaciones disruptivas para la mejora de sistemas de autenticación basados en firma y escritura manuscrita

Tesis Doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Tecnología Electrónica y de las Comunicaciones. Fecha de Lectura: 09-07-201

Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database with Automatic Labeling

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.In this study, we focus on the egocentric segmentation of arms to improve self-perception in Augmented Virtuality (AV). The main contributions of this work are: $i$ ) a comprehensive survey of segmentation algorithms for AV; $ii$ ) an Egocentric Arm Segmentation Dataset (EgoArm), composed of more than 10, 000 images, demographically inclusive (variations of skin color, and gender), and open for research purposes. We also provide all details required for the automated generation of groundtruth and semi-synthetic images; $iii$ ) the proposal of a deep learning network to segment arms in AV; $iv$ ) a detailed quantitative and qualitative evaluation to showcase the usefulness of the deep network and EgoArm dataset, reporting results on different real egocentric hand datasets, including GTEA Gaze+, EDSH, EgoHands, Ego Youtube Hands, THU-Read, TEgO, FPAB, and Ego Gesture, which allow for direct comparisons with existing approaches using color or depth. Results confirm the suitability of the EgoArm dataset for this task, achieving improvements up to 40% with respect to the baseline network, depending on the particular dataset. Results also suggest that, while approaches based on color or depth can work under controlled conditions (lack of occlusion, uniform lighting, only objects of interest in the near range, controlled background, etc.), deep learning is more robust in real AV application